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INTRODUCTION 


The  Moving  Tape  Vertical  Velocity  Indicator  (VVI)  currently  in  use  in  the  F-16C/D  aircraft  is, 
according  to  Engineering  Change  Proposal  (ECP)  1604  (General  Dynamics,  1989),  considered 
inadequate  for  rapid  assessment  of  vertical  velocity  rate  and  trend  information  required  during 
precision  approaches.  As  a  result,  the  F-16  System  Program  Office  (SPO)  is  planning  to 
incorporate  a  moving  pointer  display  as  a  replacement  for  the  moving  tape  VVI.  Due  to  limited 
space  availability,  a  standard  F-16A/B  circular  VVI  cannot  be  used.  Instead,  a  semicircular  design 
with  a  moving  pointer  is  being  considered.  In  support  of  this  effort,  the  F-16  SPO  has  tasked  the 
Crew  Station  Evaluation  Facility  (CSEF)  to  evaluate  the  acceptability  of  a  liquid  crystal, 
semicircular,  moving  pointer  VVI  as  a  potential  alternative  to  the  semicircular,  moving  pointer 
design. 

Test  Objectives 

The  primary  objective  of  this  evaluation  was  to  determine  the  acceptability  of  a  Liquid  Crystal 
Display  (LCD),  semicircular  moving  pointer  VVI  for  use  in  the  F-16.  A  secondary  goal  was  to 
compare  the  LCD  display  with  the  current  moving  tape  display  and  a  mechanical  semicircular 
display  built  by  the  CSEF.  A  total  of  four  configurations,  moving  tape,  mechanical  semicircular 
moving  pointer,  LCD  semicircular  moving  pointer  (slow  response)  and  LCD  semicircular  moving 
pointer  (fast  response),  were  evaluated;  details  on  these  configurations  are  provided  in  the  methods 
section  below. 

Specific  test  objectives  were: 

1.  Subjectively  evaluate  the  displays'  dynamics  and  operational  utility,  including  sufficiency  of 
trend  information,  acceptability  of  display  lag,  legibility  of  display  in  static  versus  dynamic 
conditions  and  the  degree  to  which  it  enhances  the  pilot's  control  of  the  aircraft. 

2.  Subjectively  evaluate  the  overall  layout  of  the  displays,  including  display  scaling,  numbering 
and  zero  point. 

3.  Obtain  pilot  inputs  regarding  an  appropriate  failure  indication  for  the  LCD  display. 

4.  Identify  any  pilot  performance  differences  across  the  displays,  including  root  mean  square 
(RMS)  vertical  velocity  deviation,  pitch  rate,  RMS  airspeed  deviation,  localizer  deviation  and 
glideslope  deviation. 

5.  Subjectively  evaluate  the  displays'  legibility,  including  brightness,  contrast,  and  design  of 
markings  and  pointers. 
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Method 


Subjects 

Ten  US AF  F-16  pilots,  stationed  at  Wright  Patterson  Air  Force  Base,  participated  as  test 
subjects.  Use  of  F-16  pilots  assured  familiarity  with  the  performance  characteristics  and  cockpit  of 
the  F-16  aircraft.  All  pilots  had  F-16A  experience  and  flight  hours  ranged  from  40  to  800  with  an 
average  of  296.  Five  pilots  also  had  F-  16C  experience;  flight  hours  ranged  from  200  through  800 
with  an  average  of  520  hours. 

Apparatus 

Crew  Station  Evaluation  Facility.  The  study  was  performed  at  the  Crew  Station  Evaluation 
Facility,  an  Air  Force  simulation  facility  that  belongs  to  the  Human  Factors  Branch  (ASD/ENECH) 
in  the  Crew  Systems  Division  (ASD/ENEC).  The  division  is  part  of  the  Support  Systems 
Engineering  Directorate  (ASD/ENE)  contained  within  the  DCS  for  Integrated  Engineering  and 
Technical  Management  (ASD/EN).  The  facility  supports  System  Program  Offices  in  their 
acquisition  engineering  through  pilot  interface  evaluations  using  man-in-the-loop  simulation. 
Currendy,  the  CSEF  has  the  capability  to  perform  full  and  pan  mission  simulations  for  a  variety  of 
aircraft  including  the  F-16.  F-l  1 1  and  C/KC-135. 

F-16  Simulator.  The  CSEF  F-16C  simulator  was  constructed  using  a  salvaged  single-seat  F-16 
cockpit,  truncated  in  front  of  the  forward  portion  of  the  windscreen,  and  approximately  57  inches 
behind  the  canopy  hinge.  The  undercarriage  was  removed,  and  the  floor  panel  section  sat  on  small 
cannister-type  wheels.  The  simulator  did  not  employ  a  motion  base.  The  cockpit  controls  and 
displays  were  configured  to  the  F-16C  Multi-National  Staged  Improvement  Program  (MSIP) 

Block  40  design  which  included  two  4x4  inch  multifunction  displays  (MFDs),  an  Integrated 
Control  Panel  (ICP),  a  Data  Entry  Display  (DED),  Hands-on  Throttle  and  Stick  (HOTAS) 
controls,  centralized  flight  instruments  and  block  40  avionics  suite.  The  head-up  display  (HUD) 
was  present  in  the  cockpit  but  was  not  functional  for  the  current  study.  The  side  control  stick, 
throttle  and  flight  controls  were  actual  F-16  components.  All  of  the  other  instruments,  controls, 
and  displays  were  simulated  using  locally  available  equipment.  A  drawing  of  the  cockpit  is 
provided  in  Figure  1. 

For  the  current  study,  all  head  down  primary  flight  instruments  were  available  to  the  pilot.  Raw 
Instrument  Landing  System  (ILS)  information  was  presented  on  the  Attitude  Director  Indicator 
(ADI),  but  no  HUD  display  was  provided.  This  configuration  was  chosen  to  force  pilot  reliance 
on  head-down  instruments.  The  remaining  displays  were  not  relevant  to  this  study. 

An  IMAGE  HIT  Visual  System  was  mounted  directly  in  front  of  the  F-16  simulator.  The 
IMAGE  HIT  system  presented  collimated  computer  generated  scenes  representing  the  outside 
world,  as  determined  by  a  data  base,  to  the  pilot.  Three  monitors  presented  a  contiguous  120* 
visual  scene  directly  in  front  of  and  to  the  left  and  right  of  the  pilot's  seated  position.  The 


2 


IMAGE  IIIT  system  had  the  capability  to  provide  a  variety  of  special  effects  including  weather, 
time  of  day,  continuous  terrain  (linked  data  bases),  texture,  and  airfield  lighting.  For  the  current 
study,  a  daytime.  Visual  Flight  Rules  scene  was  provided  during  practice  trials;  a  dense  fog  was 
provided  during  the  data  runs  to  prevent  reliance  by  the  pilot  on  the  outside  visual  scene  for  visual 
cues.  For  the  ELS  approaches  and  landings,  a  200-foot  cloud  ceiling  was  presented. 

Although  no  lighting  measurements  were  taken,  the  cockpit  was  relatively  dark  and  display 
lighting  was  required  to  read  the  VVI  and  other  flight  instruments.  The  lighting  characteristics  for 
each  VVI  are  discussed  in  more  detail  below. 

Vertical  Velocity  Indicators.  The  four  VVI  configurations  evaluated  in  the  study  are 
described  below. 

1.  F-16C/D  Moving  Tape  Design  (MT).  See  Figure  2.  This  display  is  the  current  F- 
16C/D  configuration  and  is  a  moving  tape,  fixed  pointer  design.  Two  different  scalings  are  used  in 
the  following  ranges:  0-1500  feet  per  minute  (ft/min)  (minor  graduation  marks  at  every  100 
ft/min),  and  1500-6000  ft/min  (minor  graduation  marks  at  every  500  ft/min).  Numeric  labels  are 
provided  at  the  tic  marks  associated  with  500,  1000,  1500,  2000,  3000,  4000,  5000,  and  6000 
feet  per  minute.  This  label  design  reflects  the  usefulness  of  more  accurate  information  at  small 
vertical  velocities,  such  as  those  typically  associated  with  landing;  this  detail  is  less  important  at 
larger  rates  of  climb  and  descent.  Below  the  zero  point  (descent),  white  labels  are  displayed  on  a 
dark 


Figure  2.  F- 1 6  C/D  Moving  Tape  VVI 
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background  and  above  the  zero  point  (climb),  dark  labels  are  provided  on  a  white  background. 

The  scale  is  arranged  so  that  increasing  rates  of  climb  result  in  downward  tape  movement  and 
increasing  rates  of  descent  result  in  upward  tape  movement.  The  inherent  damping  characteristics 
of  the  display  (2.2  second  first  order  lag)  were  not  altered  for  the  study.  According  to  discussions 
with  F- 16  SPO  personnel,  this  lag  was  introduced  to  prevent  jitter  in  the  display  which  is 
encountered  operationally  due  to  "noise"  in  the  signal  (e.g.  turbulence).  Display  lighting  was  not 
modified  from  that  used  in  the  aircraft;  brightness  was  generally  balanced  with  the  remaining 
cockpit  instruments  and  was  adjustable  through  the  instrument  lighting  panel. 

2.  Mechanical  Semicircular  Dial  Design  (MS).  See  Figure  3.  This  display  was  built 
locally  by  the  CSEF  to  replicate  the  anticipated  design  of  an  electromechanical,  semicircular, 
moving  pointer  design  that  is  expected  to  be  incorporated  into  the  F- 16  under  ECP  1604  (General 
Dynamics,  1989).  A  semicircular  scale  is  used  to  allow  its  placement  in  limited  cockpit  space.  The 
labeling/scaling  reflects  the  need  for  more  accurate  information  at  smaller  vertical  velocities  and  an 
effort  to  minimize  excessive  display  clutter.  This  display  was  built  for  evaluation  only  and  was  not 
intended  to  be  qualified  for  operational  use.  The  display  uses  a  nonlinear  scale.  Between  0  and 
1000  ft/min,  tic  marks  are  provided  every  100  ft/min.  Tic  marks  are  provided  every  500  ft/min 
throughout  the  rest  of  the  scale.  Spacing  between  dc  marks  is  slightly  greater  between  1000  and 
2000  ft/min  than  for  the  rest  of  the  scale.  By  modifying  the  input  signal  through  software,  a 
display  lag  equal  to  that  in  the  current  moving  tape  VVI  was  introduced  into  the  display.  This 
added  lag  controlled  for  any  effects  of  lag  differences  on  objective  and  subjective  results.  Prior  to 
the  study,  lighting  was  retrofitted  into  the  display  and  was  noticeably  unbalanced  within  the 
instrument,  but  sufficient  for  reading  the  display  in  the  ambient  lighting  conditions;  the  display 
lighting  was  not  adjustable  through  the  instrument  lighting  panel  in  the  cockpit. 


Figure  3.  Mechanical  Semicircular  VVI  (Labels  Are  White  On  Black). 
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3.  LCD  Semicircular  Dial  Design,  Slow  Response  (LCDSR).  See  Figure  4.  This 
prototype  display  was  built  by  Litton  Special  Devices.  Like  (2)  above,  it  is  a  semicircular,  moving 
pointer  design.  Display  labeling  is  painted  (white  labeling  on  dark  background)  and  the  pointer  is  a 
segmented  liquid  crystal  display.  The  pointer  moves  in  discrete  100  foot  increments  between  0  and 
1000  ft/min  and  moves  in  500  foot  increments  in  the  1000-6000  ft/min  range.  Through 
modification  of  the  software  input,  the  display  was  mechanized  to  switch  from  one  value  to 
another  at  the  midpoint  between  the  graduation  marks.  Also  through  software  modification  of  the 
display  input,  the  same  dampening  characteristics  as  the  moving  tape  configuration  were 
incorporated  to  help  control  for  effects  of  lag  on  pilot  performance.  Display  lighting  was  adjustable 
through  the  master  brightness  control  on  the  instrument  lighting  panel;  however,  the  display  was  a 
prototype  test  unit  and  no  effort  was  made  to  match  intensities  of  the  LCD  display  and  other 
cockpit  instrument  lighting  at  a  given  adjustment  setting. 


Figure  4.  Liquid  Crystal  Semicircular  Moving  pointer  VVI  (Labels  Are  White  On  Black), 

4.  LCD  Semicircular  Dial  Design,  Fast  Response  (LCDFR).  This  display  is 
identical  to  (3)  above  with  the  exception  that  it  is  mechanized  with  less  damping.  Damping 
characteristics  were  those  incorporated  by  the  manufacturer  (measured  to  be  0.8  second  first  order 
lag).  This  configuration  was  included  to  evaluate  the  effect  of  display  lag  on  pilot  performance 
with  the  LCD  format.  It  was  recognized  prior  to  the  study  that  the  final  determination  of  lag 
requires  an  actual  aircraft  signal  and  evaluation  in  turbulent  conditions. 

Experimenter's  Console.  The  experimenter's  console  included  a  complete  intercom  system 
for  four  experimenter’s/observers  and  allowed  for  communications  between  the  simulator  pilot  and 
the  experimenter.  The  console  duplicated  all  electronic  displays  (HUD,  MFDs)  and  provided 
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immediate  feedback  to  the  experimenter  on  pilot  performance.  From  the  console,  the  experimenter 
controlled  simulator  operation  and  selected  appropriate  experiment  parameters  (subject  number, 
trial  number,  experimental  condition,  etc.). 

Questionnaires.  A  set  of  Five  questionnaires  was  developed  for  the  subjective  portion  of  the 
evaluation.  Four  similar  questionnaires  specifically  addressed  each  configuration.  The  fifth 
questionnaire  allowed  the  pilot  to  direcdy  compare  the  four  configurations.  All  questionnaires  are 
included  as  Appendix  A.  Where  5  point  rating  scales  are  used,  the  descriptors  were  selected  to  be 
an  equal  interval  scale,  based  on  data  presented  in  Dyer,  et.  al.  (1976). 

Procedure 

General  Approach.  In  order  to  evaluate  the  VVI  configurations,  performance  data  were 
collected  while  having  the  pilots  perform  basic  instrument  flight  maneuvers  (rate  climbs  and 
descents)  and  an  approach/landing.  Each  pilot  flew  four  sets  of  simulator  runs  (one  for  evaluating 
each  configuration).  Immediately  prior  to  each  mission,  the  pilot  was  provided  with 
training/familiarization  on  the  configuration  under  evaluation. 

Evaluation  Maneuvers.  The  flight  maneuvers  selected  for  this  evaluation  were  VVI-intensive 
and  allowed  a  thorough  evaluation  of  each  configuration.  The  approach/landing  was  included  as  a 
realistic  application  of  the  VVI.  Three  climb/descent  rates  were  used  (750,  1500  and  3000  ft/min). 
These  rates  of  climb  and  descent  were  selected  to  allow  an  evaluation  of  the  various  display 
scalings  and  provide  tasks  of  varying  difficulty.  The  following  maneuvers  were  used  in  the 
evaluation: 

1.  Straight  and  level  flight  for  45  seconds  while  maintaining  10000  ft,  350  knots  and  herding 
of  268°. 

2.  Rate  descent  from  10000  to  6000  feet,  maintaining  350  knots,  3000  ft/min,  and  heading  of 
268'. 

3.  Rate  climb  from  6000  to  10000  feet,  maintaining  350  knots,  3000  ft/min,  and  heading  of 
268*. 

4.  Rate  climb  from  8000  to  10000  feet,  maintaining  350  knots,  1500  ft/min,  and  heading  of 
268*. 

5.  Rate  descent  from  10000  to  8000  feet,  maintaining  350  knots,  1500  ft/min  rate  of  descent, 
and  heading  of  268°. 

6.  Rate  climb  from  9000  to  10000  feet,  maintaining  150  knots,  750  ft/min,  and  heading  of 
268'  (landing  gear  down). 
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7.  Rate  descent  from  10000  to  9000  feet,  maintaining  150  knots,  750  ft/min,  and  heading  of 
268*  (landing  gear  down). 

8.  ILS  Approach  and  Landing  at  Ramstein  AB.  From  reset,  the  pilot  was  on  a  heading  of  222, 
altitude  3200  feet,  200  knots  at  12  DME  with  landing  gear  down.  Upon  interception  of  the 
localizer,  the  pilot  initiated  a  45°  turn  to  intercept  final.  The  pilot  was  instructed  to  fly  a  final 
approach  speed  of  145  knots  or  1 1*  Angle  of  Attack. 

Test  Procedures.  All  pilots  were  given  the  same  introduction  and  training  briefing  which 
included  background  on  the  study,  descriptions  of  all  configurations,  and  pilot  tasks  to  be 
performed  during  the  simulation.  Prior  to  beginning  data  collection,  each  pilot  was  given  10 
minutes  to  become  familiar  with  relevant  controls  and  displays.  After  familiarization,  each  pilot 
flew  a  practice  DLS  approach  and  landing  at  Ramstein  AB. 

For  each  VVI,  the  following  procedure  was  followed.  The  test  engineer  described  the 
configuration  and  dynamics  of  the  VVI  configuration  to  be  tested  and  instructed  the  pilot  to  adjust 
display  lighting  to  a  comfortable  level  (except  for  the  MS  which  did  not  have  adjustable  lighting). 
The  pilot  then  performed  three  practice  rate  climbs/descents  that  were  representative  of  the 
maneuvers  to  be  performed  for  the  data  collection  trials;  maneuvers  2, 4  and  7,  randomly  ordered 
for  each  pilot,  were  used  for  this  purpose.  All  practice  trials  were  performed  in  VFR  conditions 
with  unlimited  visibility. 

After  completing  the  practice  trials,  all  eight  maneuvers  were  performed  for  data  collection.  For 
each  one,  the  test  engineer  described  the  maneuver,  instructed  the  pilot  to  properly  configure  the 
simulator  (i.e.,  gear  up  or  down  as  appropriate,  instrument  mode  in  ILS  or  NAV,  as  appropriate) 
and  reset  the  simulator.  The  simulator  was  automatically  set  to  the  proper  altitude  and  airspeed  for 
the  maneuver  being  performed.  Upon  releasing  the  simulator,  the  test  engineer  announced,  “You 
are  flying"  and  the  pilot  immediately  began  the  maneuver.  Upon  reaching  his  target  altitude,  the 
simulator  automatically  reset  when  the  flight  path  of  the  simulator  was  within  3’  of  straight  and 
level  for  3  seconds  and  simulator  altitude  was  within  50  feet  of  the  target  altitude  for  the  task.  For 
the  ILS  approaches,  the  test  engineer  manually  reset  the  simulator  after  the  aircraft  was  on  the 
ground. 

The  order  of  VVI  presentation  was  counterbalanced  to  compensate  for  training  effects.  The 
order  of  performance  of  the  maneuvers  for  each  VVI  was  randomized  with  the  constraint  that  the 
ILS  approach  and  landing  was  last.  All  rate  climb/descent  data  collection  runs  were  performed  in 
Instrument  Flight  Rules  (IFR)  conditions  with  zero  visibility.  A  200  foot  cloud  ceiling  was 
simulated  for  the  ILS  approaches  and  landings. 

After  flying  with  each  configuration,  the  pilot  completed  the  appropriate  subjective  questionnaire 
and  took  a  short  break  while  the  next  configuration  was  installed.  After  performance  of  all  tasks 
for  the  last  configuration,  the  pilot  completed  the  comparison  questionnaire  and  was  debriefed. 
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Performance  Data  Collection 


Rate  Climbs  and  Descents.  During  rate  climbs  and  descents,  the  following  data  were 
collected: 

a.  Vertical  velocity  deviation  (feet  per  minute).  The  Root  Mean  Square  (RMS)  deviation  from 
the  specified  rate  of  climb  or  descent  was  collected. 

b.  Airspeed  deviation  (knots).  The  RMS  deviation  from  the  specified  airspeed  was  collected 
throughout  each  rate  climb  and  descent.  A  VVI  that  is  more  difficult  to  interpret  may  result  in 
greater  attention  being  allocated  to  the  VVI  and,  consequently,  less  attention  being  allocated  to 
airspeed.  Under  these  conditions,  a  greater  airspeed  deviation  may  occur. 

c.  Pitch  Rate  (degrees  per  second).  The  rate  of  change,  at  a  given  time,  of  aircraft  pitch  in 
degrees  per  second.  Pitch  rate  can  provide  an  indication  of  workload  (i.e.  stick  inputs  to  correct 
pitch)  associated  with  the  display. 

All  data  were  collected  at  a  rate  of  5  Hertz  during  the  entire  length  of  the  maneuver.  All  these 
data  were  used  for  the  analysis  of  the  RMS  airspeed  and  pitch  rate  measures.  However,  the  RMS 
vertical  velocity  deviation  data  were  only  valid  while  the  pilot  was  attempting  to  maintain  a  constant 
rate  of  climb  or  descent.  An  examination  of  the  raw  data  showed  that  pilots  were  maintaining  a 
constant  rate  of  climb/descent  during  the  middle  third  (based  on  altitude)  of  the  maneuvers. 
Therefore,  only  data  from  the  middle  portion  of  the  rate  climbs/descents  were  extracted  for  use  in 
the  analysis  of  RMS  vertical  velocity  deviation.  For  the  level  flight  task,  pilots  used  the  first  15 
seconds  of  flight  to  stabilize  the  aircraft  and  establish  straight  and  level  flight  at  10000  feet.  The 
first  15  seconds  were  therefore  not  analyzed. 

ILS  Approach  and  Landing.  The  following  performance  measures  were  collected  during  the 
ELS  approach  and  landings,  from  glideslope  interception  until  altitude  reached  200  feet. 

1.  RMS  glideslope  deviation  (degrees).  The  number  of  degrees  of  vertical  deviation  from  the 
glide  path  (2.5*)  was  collected  to  determine  any  enhancement  in  vertical  control  offered  by  any  one 
of  the  four  VVI  configurations. 

2.  RMS  localizer  deviation  (degrees).  The  number  of  degrees  of  horizontal  deviation  from  the 
runway  centerline  was  collected  and  used  as  an  overall  measure  of  pilot  performance.  Variation  in 
the  variable  was  not  considered  to  be  influenced  by  vertical  velocity. 

3.  Pitch  rate.  The  rate  of  change,  at  a  given  time,  of  aircraft  pitch  in  degrees  per  second.  Pitch 
rate  can  provide  an  indication  of  workload  (i.e.  stick  inputs  to  correct  pitch)  produced  by  the 
display. 
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4.  Airspeed  deviation  (knots).  RMS  airspeed  deviation  from  the  preferred  approach  speed  (145 
knots)  was  collected.  As  stated  previously,  greater  attention  allocated  to  other  displays  (ILS  data, 
altimeter,  ADI,  VVI)  would  result  in  less  attention  devoted  to  maintaining  the  specified  airspeed. 

Subjective  Data  Collection 

Pilots  completed  questionnaires  specific  to  the  configuration  under  evaluation  immediately  after 
performing  the  data  collections  tasks.  These  questionnaires  addressed  such  issues  as  legibility  of 
various  display  features,  display  dynamics  and  operational  utility  and  are  provided  as  Appendix  A. 
Pilot  comments  were  also  collected  and  are  included  as  Appendix  B. 
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RESULTS 

Performance  Data  Analysis 

The  data  from  the  rate  climbs/descents  (maneuvers  1  through  7)  were  analyzed  separately  from 
the  ELS  approach  data  (maneuver  8).  For  the  rate  climbs  and  descents,  all  data  were  reduced  to  a 
single  average  value  for  each  pilot,  VVI  and  maneuver.  RMS  airspeed  deviation  and  pitch  rate 
were  analyzed  as  a  function  of  the  four  VVI  configurations  and  the  six  rate  climbs/descents 
(maneuvers  2  though  7)  in  a  4  x  6  Analysis  of  Variance  (ANOVA).  Similarly,  the  RMS  vertical 
velocity  deviation  data  were  analyzed  as  a  function  of  the  four  WI  configurations  and  seven 
maneuvers  (the  six  rate  climbs/descents  and  the  level  flight  maneuver)  in  a  4  x  7  repeated  measures 
ANOVA.  For  the  ILS  runs,  the  data  were  also  reduced  to  a  single  value  for  each  pilot  and  VVI. 
Glideslope  deviation,  localizer  deviation,  airspeed  deviation  and  pitch  rate  were  examined  as  a 
function  of  the  four  VVI  configurations  in  one-way  repeated  measures  ANOVAs.  An  alpha  level 
of  less  than  0.05  was  used  as  the  criteria  for  rejection  of  the  null  hypothesis  in  all  of  the  analyses. 

Results  For  Rate  Climbs  and  Descents 

RMS  Airspeed  Deviation.  The  results  of  the  Airspeed  Deviation  ANOVA  are  presented  in 
Table  1 .  No  main  effects  for  VVI,  maneuver  or  their  interaction  were  found.  Average  RMS 
airspeed  deviations  for  different  VVIs  and  different  maneuvers  are  shown  in  Figures  5  and  6. 


Table  1.  ANOVA  Table  for  RMS  Airspeed  Deviation. 


Source 

dL 

Si 

F 

e 

WI 

3 

111.35 

1.65 

0.2007 

Maneuver 

5 

111.78 

0.47 

0.7980 

VVI  x  Maneuver 

15 

761.79 

1.12 

0.3460 

Pilot 

9 

3666.01 

Pilot  x  VVI 

27 

606.52 

Pilot  x  Maneuver 

Pilot  x  Maneuver 

43 

2054.52 

x  VVI 

128 

5809.12 

1 1 


1  0 


Moving  Mach.  LCD(SR)  LCD(FR) 
Tape  Semi. 

VVI 

Figure  5.  Average  Airspeed  Deviation  for  Different  VVIs 
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Figure  6.  Average  Airspeed  Deviation  for  Different  Maneuvers 
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Pitch  Rate.  Table  2  shows  the  results  for  the  pitch  rate  as  a  function  of  VVI,  Maneuver,  and 
their  interaction.  Significant  effects  were  found  for  VVI  and  Maneuver,  average  pitch  rates  for 
different  VVIs  are  shown  in  Figure  7.  The  post-hoc  Duncan  test  indicated  that  pitch  rate  for  MT 
was  significantly  greater  than  that  for  MS,  LCDSR  and  LCDFR;  the  latter  three  did  not  differ  from 
each  other.  Average  pitch  rates  for  different  maneuvers  are  shown  in  Figure  8.  The  post-hoc 
Duncan  test  showed  the  750  ft/min  runs  yielded  significantly  greater  pitch  rates  than  all  of  the  other 
maneuvers.  No  interaction  was  found  between  VVI  and  Maneuver. 
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Table  2.  ANOVA  Table  for  Pitch  Rate 


Source 

dL 

5S 

F 

WI 

3 

8.01 

4.51 

Maneuver 

5 

24.21 

11.11 

VVI  x  Maneuver 

15 

7.98 

1.28 

Pilot 

9 

18.84 

Pilot  x  VVI 

27 

15.97 

Pilot  x  Maneuver 

43 

18.74 

Pilot  x  Maneuver 
x  WI 

128 

53.10 

Moving  Mech.  LCD(SR)  LCD(FR) 
Tape  Semi. 


VVI 


Figure  7.  Pitch  Rate  for  Different  VVls. 
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Figure  8.  Pitch  Rate  for  Different  Maneuvers. 
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RMS  Vertical  Velocity  Deviation.  For  RMS  vertical  velocity  deviation,  significant  main 
effects  were  found  for  both  VVI  and  Maneuver;  the  ANOVA  results  are  shown  in  Table  3.  Post 
hoc  tests  showed  that  RMS  vertical  velocity  deviation  was  significandy  smaller  for  the  LCDFR 
than  the  MT  and  LCDSR;  performance  on  maintaining  the  specified  vertical  velocity  with  MS  did 
not  significandy  differ  from  any  of  the  other  displays.  Average  RMS  vertical  velocity  deviation  is 
shown  in  Figure  9. 


Table  3.  ANOVA  Table  for  RMS  Vertical  Velocity  Deviation 


Source 

df. 

££ 

F 

e 

WI 

3 

458625 

3.39 

0.0322 

Maneuver 

6 

8323082 

40.75 

0.0001 

VVI  x  Maneuver 

18 

783124 

1.02 

0.4362 

Pilot 

9 

1890313 

Pilot  x  VVI 

27 

1216952 

Pilot  x  Maneuver 

52 

1770334 

Pilot  x  Maneuver 
x  WI 

155 

6585414 

RMS 
Vertical 
Velocity 
Deviation 
(feet  per 
minute) 


Moving  Mach.  LCO(SR)  LCO(FR) 

Taps  Ssmi. 


Wl 


Figure  9.  RMS  Vertical  Velocity  Deviation  Across  VVIs 


The  750  ft/min  climb  yielded  the  best  performance  in  maintaining  the  desired  vertical  velocity 
(see  Figure  10).  The  1500  ft/min  maneuvers  did  not  differ  from  each  other,  but  yielded  worse 
performance  than  all  other  maneuvers.  Performance  on  the  remaining  maneuvers  did  not  differ 
from  each  other.  The  interaction  between  VVI  and  Maneuver  was  not  statistically  significant. 
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Figure  10.  RMS  Vertical  Velocity  Deviation  Across  Maneuvers 


Results  For  ILS  Approaches  and  Landings 

Tables  4,  5,  6  and  7  show  the  results  of  ANOVAs  for  glideslope  deviation,  localizer  deviation, 
pitch  rate  and  RMS  airspeed  deviation.  No  statistically  significant  differences  were  found  for  any 
of  the  performance  measures  across  the  different  VVIs.  Average  performance  on  glideslope 
deviation,  localizer  deviation,  RMS  airspeed  deviation  and  pitch  rate  for  each  VV1  is  shown  in 
Figures  11,  12,  13,  and  14,  respectively. 


Table  4.  ANOVA  Table  for  Glideslope  Deviation  During  the  ILS  Approach. 


Source 

dL 

S£ 

F 

C 

Wl 

3 

0.003 

1.70 

0.  8350 

Pilot 

9 

0.032 

VVI  x  Pilot 

27 

0.090 

Table  5.  ANQVA  Table  for  Localizer  Deviation  During  the  ILS  Approach. 


Source 

dL 

SI 

F 

J2 

VVI 

3 

0.01 

0.60 

0.6183 

Pilot 

9 

0.26 

VVI  x  Pilot 

27 

0.18 

15 


F 

0.69 


E 

0.5640 


Source 

dL 

WI 

3 

65.18 

Pilot 

9 

1779.54 

WI  x  Pilot 

27 

845.88 

Table  7.  ANOVA  Table  for  Pitch  Rate  During  the  ILS  Approach. 


Source 

dL 

E 

WI 

3 

0.14 

0.28 

Pilot 

9 

15.33 

WI  x  Pilot 

27 

4.60 

E 

0.8378 
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Deviation 

(degrees) 
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Subjective  Data 

The  questionnaires  addressed  two  general  areas,  (1)  display  design,  layout  and  legibility,  and 
(2)  operational  utility.  No  formal  statistical  analyses  were  performed  on  the  data  but  frequency 
distributions  were  developed  for  all  questions  with  nominal  scale  responses.  Some  questions  were 
rated  on  a  score  of  1  through  5  (Completely  Unacceptable  =  1,  Completely  Acceptable  =  5). 
Average  ratings  were  calculated  for  each  configuration. 

Display  Design  and  Layout.  As  shown  in  Figures  15,  16  and  17,  the  majority  of  pilots  felt 
that  label  size,  graduation  mark  size,  and  graduation  mark  spacing  were  about  right  for  all  displays. 
A  small  percentage  of  pilots  felt  that  these  features  were  too  large  for  the  MT  display,  and  that  these 
features  were  too  small  for  the  LCD  and  MS  display. 


Y  Too  Small  About  Too  Large 
Right 


Label  Size 


Figure  15.  Pilots  assessment  of  number  label  size  for  the  VVI  Configurations 


Figure  16.  Pilots  subjective  assessment  of  graduation  mark  size 
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Too  About  Too 
Small  Right  Large 


Ratings  of  Graduation  Mark  Spacing 


Figure  17.  Riots'  Subjective  Assessment  of  Graduation  Mark  Spacing 


As  shown  in  Figures  18  and  19,  average  ratings  of  label  location  and  the  acceptability  of  the 
zero  point  on  each  display  were  very  high  (between  moderately  and  completely  acceptable). 
Pointer  legibility  was  given  a  high  rating  for  both  moving  pointer  displays  (Figure  20).  Pointer 
mechanization  was  rated,  on  average  4.4  (i.e.,  between  moderately  and  completely  acceptable)  for 
the  LCD  display,  suggesting  that  the  discrete  movement  of  the  LCD  pointer  display  was 
acceptable. 
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Vertical  Velocity  Indicator 


5  =  Completely  Acceptable 
4  =  Moderately  Acceptable 
3  =  Borderline 

2  =  Moderately  Unacceptable 
1  =  Completely  Unacceptable 


Figure  18.  Acceptability  of  Location  of  Number  Labeling  on  Scale 
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Moving  Mech.  LCD 
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5  =  Completely  Acceptable 
4  =  Moderately  Acceptable 
3  =  Borderline 

2  =  Moderately  Unacceptable 
1  =  Completely  Unacceptable 


Figure  19.  Acceptability  of  the  Zero  Indication 
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Vertical  Velocity  Indicator 


Figure  20.  Acceptability  of  Pointer  Legibility 


Acceptability  of  display  contrast  for  the  three  displays  is  shown  in  Figure  21 .  Only  one  pilot 
indicated  that  contrast  was  too  low  on  the  LCD  display.  All  pilots  felt  the  MS  and  MT  displays  had 
sufficient  contrast.  In  the  relatively  dark  ambient  lighting  conditions  in  the  simulator,  all  pilots  felt 
that  the  LCD  brightness  could  be  set  to  a  comfortable  level.  Nine  pilots  felt  that  the  MT  could  be  set 
to  a  comfortable  brightness. 


20 


Frequency 


1  "  - 


B  Moving  Tape 
9  Mech.  Semi. 
B  LCD 


Too  About  Too 
Low  Right  High 


Contrast 


Figure  21.  Pilots'  Subjective  Assessment  of  Contrast  Between  Marking/Pointer  and  Display 

Background 


Operational  Utility. 

Figure  22  shows  pilots'  ratings  of  the  overall  scaling  of  the  display.  All  displays  were  rated 
above  “borderline,"  with  the  MS  display  having  the  highest  average  rating  and  the  MT  display 
having  the  lowest.  Comments  indicated  that  pilots  would  prefer  more  detail  in  the  MT  display 
between  0  and  1500  ft/min. 


Average 

Rating 


5  =  Completely  Acceptable 
4  =  Moderately  Acceptable 
3  =  Borderline 

2  =  Moderately  Unacceptable 
I  =  Completely  Unacceptable 


Moving  Mech.  LCD 

Tape  Semi. 


Vertical  Velocity  Indicator 


Figure  22.  Acceptability  of  Display  Scaling 
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As  shown  in  Figures  23  and  24,  the  ability  to  quickly  read  the  displayed  value,  and  ease  of 
interpretation  under  static  conditions  were  rated,  on  average,  above  moderately  acceptable  for  all 
displays. 


Moving  Mach.  LCO<FR)  LCD(SR) 
tape  Semi. 


5  =  Completely  Acceptable 
4  =  Moderately  Acceptable 
3  =  Borderline 

2  =  Moderately  Unacceptable 
1  =  Completely  Unacceptable 


Vertical  Velocity  Indicator 


Figure  23.  Average  Ratines  of  the  Ability  to  Quickly  Read  the  Display 


Moving  Mech.  LCD(FR)  LCD(SR) 
Tape  Semi. 


Completely  Acceptable 
Moderately  Acceptable 
Borderline 

Moderately  Unacceptable 
Completely  Unacceptable 


Vertical  Velocity  Indicator 


Figure  24.  Ease  of  Interpreting  the  Displays  Under  Static  Conditions 


Figures  25,  26  and  27  all  show  a  similar  trend  across  the  four  configurations.  The  MS  is 
consistently  the  most  preferred  for  ease  of  interpretation  under  changing  conditions,  quality  of 
trend  information  and  ability  to  enhance  aircraft  control.  The  MT  display  is  consistently  the  least 
preferred.  Responses  to  the  acceptability  of  display  lag  were  mixed  across  the  four  configurations. 
The  subjective  results  are  shown  in  Figure  28. 
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Moving  Mech.  LCD(FR)  LCD(SR) 
Tape  Semi. 


Vertical  Velocity  Indicator 


5  =  Completely  Acceptable 
4  =  Moderately  Acceptable 
3  =  Borderline 

2  =  Moderately  Unacceptable 
1  =  Completely  Unacceptable 


Figure  25.  Ease  of  Interpretation  Under  Changing  Conditions 


Moving  Mech.  LCD(FR)  ICD(SR) 
Tape  Semi. 


5  =  Completely  Acceptable 
4  =  Moderately  Acceptable 
3  =  Borderline 

2  =  Moderately  Unacceptable 
1  =  Completely  Unacceptable 


Vertical  Velocity  Indicator 


Figure  26.  Acceptability  of  Trend  Information  Provided  by  the  Display 


Moving  Mech.  LCD(FR)  LCD(SR) 
Tape  Semi. 


5  =  Completely  Acceptable 
4  =  Moderately  Acceptable 
3  =  Borderline 

2  =  Moderately  Unacceptable 
1  =  Completely  Unacceptable 


Vertical  Velocity  Indicator 


Figure  27.  Subjective  Assessment  of  Display's  Ability  to  Enhance  Aircraft  Control 


F 


Slow  Right 


Subjective  Assessment  of  Response  Lag 
Figure  28.  Subjective  Assessment  of  Acceptability  of  Response  Lag 

Overall  Ratings  of  the  Displays.  Figure  29  shows  the  overall  ratings  of  the  displays  for  use 
in  the  F-16;  average  ratings  show  the  MT  to  be  the  least  preferred  with  a  slight  preference  for  the 
MS  display  above  the  LCDFR,  and  a  slight  preference  for  LCDFR  above  LCDSR.  Median 
rankings  of  the  displays  were  consistent  with  Figure  24.  Median  rank  of  the  displays  (1  =  most 
preferable,  4  =  least  preferable)  on  the  basis  of  ease  of  interpretation  and  overall  preference  were  1, 
2.5,  2.5  and  4  for  MS,  LCDFR,  LCDSR  and  MT,  respectively. 


Moving  Mech.  LCO(FR)  LCD(SR) 
Tape  Semi. 


Completely  Acceptable 
Moderately  Acceptable 
Borderline 

Moderately  Unacceptable 
Completely  Unacceptable 


Vertical  Velocity  Indicator 


Figure  29.  Overall  Acceptability  for  Use  in  the  F-16 


Other  results  from  the  questionnaires  include  the  following.  Eight  pilots  felt  that  the  MT 
displayed  sufficient  precision;  nine  pilots  felt  that  the  MS  and  LCD  displays  provided  sufficient 
precision.  About  half  of  the  pilots  felt  that  the  tic  mark  at  -750  ft/min  on  the  MT  display  was  useful 
during  approaches  and  landings  (two  did  not  notice  the  mark)  and  felt  that  a  tic  mark  at  -750  would 
be  useful  on  the  MT  and  LCD  displays.  Comments  indicated  that  a  750  ft/min  rate  of  descent  is 
not  appropriate  for  many  landing  configurations.  When  asked  what  would  be  a  good  failure 
indication  for  the  LCD  display,  six  pilots  suggested  the  removal  of  the  pointer  from  the  display. 
Other  responses  included:  an  “X"  displayed  on  the  face,  off  flag,  actuation  of  several  pointers  for  a 
“fan"  indication  and  an  INS  failure  indication. 
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Pilot  Comments.  Highlights  from  the  pilot  comments  are  provided  below. 


a.  Three  pilots  commented  that  trend  information  is  difficult  to  obtain  from  the  MT  display. 

b.  Four  pilots  felt  that  the  MS  VVI  was  a  good  design. 

c.  Two  pilots  commented  that  the  LCDSR  was  better  than  the  LCDFR  for  approaches. 

d.  Two  pilots  did  not  like  the  ratcheting  effect  of  the  LCD  pointer  and  found  it  distracting. 
Two  pilots  felt  the  ratcheting  attracted  attention  to  a  change  in  the  periphery  and  provided  a  very 
positive  indication  of  change. 

e.  Three  pilots  stated  that  the  displays  must  be  evaluated  in  turbulent  conditions  before  a  final 
determination  can  be  made  for  lag. 
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DISCUSSION 


Vertical  Velocity  Indicators 

Although  the  RMS  airspeed  and  RMS  vertical  velocity  deviation  data  showed  no  performance 
difference  between  the  MT,  MS  and  LCDSR  displays,  the  greater  pitch  rate  yielded  by  the  MT 
display  indicates  a  performance  benefit  with  the  moving  pointer  displays  (assuming  response  lag  is 
held  constant).  These  results  can  be  explained  in  terms  of  several  human  factors  design 
deficiencies  associated  with  the  MT  display. 

According  to  Wickens  (1984)  a  display  is  optimal  if  it  presents  information  in  a  format  that  is 
compatible  with  the  user's  internal  mental  representation  of  the  information  being  displayed; 
following  this  principle  ensures  the  minimum  amount  of  mental  translation  required  in  reading  the 
display.  It  is  reasonable  to  assume  that  the  pilot  conceptualizes  vertical  velocity  as  a  continuously 
varying  quantity  where  increases  are  associated  with  upward  vertical  aircraft  movement  and 
decreases  are  associated  with  downward  vertical  aircraft  movement.  This  suggests  that  a  vertical 
velocity  indicator  should  have  the  following  design  characteristics:  analog  display  movement; 
vertical  orientation;  a  scale  ordered  so  that  the  upper  portion  of  the  scale  represents  climbing  by 
greater  amounts  and  the  lower  portion  of  the  scale  corresponds  to  descent;  and  display  movement 
such  that  upward  motion  indicates  a  climb  and  downward  motion  indicates  descent. 

While  the  moving  pointer  display  designs  generally  conform  to  these  guidelines,  it  is  impossible 
to  do  so  with  the  moving  tape  display.  Because  of  the  labeling  of  the  scale,  increasing  rates  of 
climb  result  in  a  downward  movement  of  the  display.  Reversing  the  scale  to  allow  upward  tape 
movement  to  indicate  climb  would  require  reversal  of  the  display  labeling,  in  which  the  lower, 
rather  than  upper  portion  of  the  scale  would  indicate  greater  rates  of  climb. 

Only  a  portion  of  the  scale  on  the  MT  is  visible  at  any  given  time,  requiring  the  pilot  to  read  the 
values  shown,  determine  his  location  on  the  scale  (for  ascent  or  descent),  and  read  the  values 
relative  to  the  pointer.  In  the  moving  pointer  designs,  the  entire  scale  is  visible  at  all  times,  which 
simplifies  reading  the  display  by  reducing  the  need  to  read  specific  values;  a  general  indication  of 
vertical  velocity  can  be  easily  obtained  from  pointer  position.  The  moving  tape  display  is  more 
difficult  to  read  than  the  fixed  scale  designs  in  dynamic  conditions  because  of  scale  movement. 

Not  only  does  this  cause  difficulty  reading  the  actual  vertical  velocity  at  a  given  time,  but  also 
increases  the  difficulty  of  extracting  trend  information  from  the  display. 

These  factors  combined  result  in  a  display  that  lends  itself  to  reading  errors,  increased  time 
required  to  interpret  the  display,  and  greater  difficulty  of  reading  the  display  under  changing 
conditions.  All  of  these  effects  could  have  contributed  to  increased  pitch  rates,  by  requiring  a 
greater  reading  time  and  increasing  the  potential  for  display  reading  errors. 
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Effect  of  Response  Lag 

Response  time  of  the  display  did  not  impact  pitch  rate  (i.e.,  LCDSR  and  LCDFR  did  not  differ 
on  the  pitch  rate  measure)  but  did  influence  vertical  velocity  deviation.  A  faster  response  time  of  the 
VVI  provides  a  more  precise  display,  provides  more  accurate  trend  information,  and  reduces  the 
pilot's  reliance  on  estimated  vertical  velocity  at  a  given  time.  As  expected,  performance  improved 
with  the  LCDFR  over  the  LCDSR.  This  same  effect  would  be  expected  with  the  Mechanical 
semicircular  display,  if  its  response  were  increased.  However,  this  improved  performance  with  a 
faster  response  cannot  be  generalized  to  the  moving  tape  display  with  the  same  level  of  confidence, 
because  increasing  the  speed  of  scale  movement  may  reduce  its  legibility.  Note  that  the 
determination  of  display  lag  should  be  made  after  evaluation  in  turbulent  conditions  with  the 
characteristically  “noisy"  VVI  signal  under  turbulent  conditions.  The  current  study  results  can 
only  provide  an  assessment  of  performance  with  different  lags.  It  cannot  determine  what  lag  is 
acceptable  under  actual  conditions. 

ILS  Data 

As  was  stated  previously,  no  objective  data  differences  were  found  during  any  of  the  ILS 
approaches.  The  lack  of  statistically  significant  differences  is  probably  due  to  the  role  of  the  VVI 
during  an  approach  and  landing.  During  this  phase  of  flight,  the  VVI  is  used  primarily  as  a  cross¬ 
check  instrument  which  may  assist  in  maintaining  alignment  with  the  glideslope  and  localizer  bars 
rather  than  maintaining  a  specified  VVI.  The  objective  data  measures  were  much  more  sensitive  to 
differences  across  VVIs  in  the  rate  climbs  and  descents,  which  were  VVI  intensive  tasks. 

Differences  Across  Maneuvers 

The  main  effect  for  maneuver  on  the  pitch  rate  measure  showed  that  the  750  ft/min  runs 
required  higher  levels  of  workload  than  any  of  the  other  rate  climb/descent  maneuvers.  However, 
vertical  velocity  deviation  showed  improved  accuracy  at  maintaining  the  specified  vertical  velocity 
under  the  same  maneuvers.  The  increased  precision  provided  by  all  of  the  displays  in  the  750 
ft/min  range  may  have  contributed  to  the  improved  accuracy  at  maintaining  750  ft/min.  This 
greater  display  accuracy,  being  more  sensitive  to  deviations  from  the  specified  rate  of  climb  or 
descent,  may  have  resulted  in  more  pitch  corrections  being  input  by  the  pilot. 

The  greater  vertical  velocity  deviation  found  for  the  1500  ft/min  maneuvers  was  unexpected.  On 
all  of  the  displays,  the  1500  ft/min  graduation  mark  was  at  or  near  the  transition  point  from  one 
scaling  to  another.  Given  a  constant  rate  of  change  of  vertical  velocity,  the  speed  of  pointer 
movement  changes  as  it  crosses  from  one  scaling  to  another.  This  noncontinuity  in  pointer 
movement  may  have  increased  the  difficulty  of  obtaining  trend  information  when  approaching  the 
1 500  ft/min  mark,  resulting  in  lower  accuracy. 
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Subjective  Data 

The  subjective  data  supported  the  performance  data.  General  display  design  and  layout 
features  as  well  as  the  precision  provided  were  generally  acceptable  to  most  pilots  for  all  displays. 
Differences  in  preferences  were  more  evident  when  pilots  were  asked  about  display  dynamics  and 
trend  information,  on  which  the  MT  was  least  preferred.  There  also  appeared  to  be  a  slight  but 
consistent  trend  of  preference  for  the  MS  display  over  the  LCD  displays  for  use  under  dynamic 
conditions,  quality  of  trend  information,  and  ability  to  enhance  aircraft  control.  Furthermore, 
overall  ratings  and  rankings  showed  the  MS  to  be  the  most  preferred  and  the  MT  to  be  the  least 
preferred,  with  both  LCD  configurations  falling  in  between.  This  pattern  of  results  may  be  due  to 
the  discrete  pointer  movement  on  the  LCD,  which  causes  a  slight  degradation  in  the  display  of 
trend  information. 

These  subjective  results  are  in  agreement  with  a  previous  similar  study  (Dudley,  Hassoun  and 
Gavem,  1988),  in  which  a  landing  task  was  used  to  evaluate  performance  and  subjective 
differences  between  a  moving  pointer,  vertical  scale  VVI  and  the  current  F-16C/D  moving  tape 
VVI.  Although  no  performance  differences  were  found,  pilots  preferred  the  moving  pointer 
display  over  the  moving  tape  configuration. 

The  subject  evaluation  of  acceptability  of  response  lag  provided  interesting  results.  Even  though 
the  MT,  MS  and  LCDSR  had  identical  lags,  six  pilots  indicated  the  MT  response  was  too  slow,  six 
pilots  indicated  the  LCDSR  response  was  about  right,  and  nine  pilots  responded  that  the  MS 
response  was  about  right.  All  of  the  pilots  felt  the  LCDFR  response  was  about  right.  These 
results  suggest  that  different  lags  may  be  optimal  for  different  display  configurations.  In 
particular,  a  greater  lag  may  be  acceptable  in  a  display  that  provides  high  quality  trend  information 
(such  as  the  MS  display)  than  is  acceptable  in  a  display  that  provides  relatively  poor  trend 
information  (e.g.,  the  MT  display). 

Of  specific  interest  to  the  LCD  display  were  pointer  legibility,  pointer  mechanization,  contrast, 
and  the  ability  to  set  the  brightness  to  a  comfortable  level.  The  LCD  pointer  moves  in  discrete 
steps  across  the  display  scale,  rather  than  in  a  continuous  fashion.  Therefore,  trend  information 
and  display  accuracy  are  both  slightly  degraded.  As  discussed  above,  this  characteristic  of  the 
LCD  display  appeared  to  have  a  small  influence  on  the  subjective  ratings  of  interpretability  in 
dynamic  conditions.  However,  the  majority  of  pilots  indicated  that  the  LCD  displayed  sufficient 
accuracy  and  that  the  pointer  mechanization  was  acceptable.  Display  contrast  and  brightness 
adjustments  were  both  acceptable  as  currently  designed. 

Other  Results 

In  addition  to  addressing  the  goals  described  previously,  the  study  provided  an  opportunity  to 
identify  potential  problems  with  the  prototype  LCD  display  design  that  would  need  to  be  addressed 
prior  to  its  incorporation  into  an  aircraft.  The  prototype  unit  used  in  the  study  exhibited  two 
problems.  Occasionally,  several  pointers  would  actuate  simultaneously,  causing  a  fanning  effect 
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on  the  display  that  obscured  the  actual  displayed  value.  Second,  the  pointer  would  sometimes 
disappear  in  certain  portions  of  the  display  during  operation.  Through  in-house  troubleshooting 
and  discussions  with  Litton  personnel,  it  appeared  that  the  physical  stress  on  the  display  housing 
(when  mounted  into  the  cockpit)  may  have  been  the  cause  of  the  problems.  Litton  has  been  able  to 
reproduce  the  effect  on  other  test  items  and  is  working  towards  a  solution. 

Generalizing  Current  Results  to  the  Operational  Setting 

The  current  study  was  designed  to  be  VVI-intensive  in  order  to  be  sensitive  to  the  effects  of 
different  displays  on  performance.  As  a  result,  these  same  performance  differences  may  not  be 
realized  with  the  typical  operational  application  of  the  VVI,  where  it  is  primarily  used  as  a  cross¬ 
check  instrument,  as  an  alternative  to  the  HUD  vertical  velocity  display,  or  in  the  case  of  a  HUD 
failure.  The  lack  of  significant  differences  during  the  ILS  approaches  exemplifies  this  issue. 
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CONCLUSIONS 


The  current  study  found  a  performance  decrement  for  the  current  moving  tape  display  when 
the  display  response  is  held  constant;  various  human  factors  design  deficiencies  contributed  to  this 
result.  No  differences  between  the  various  moving  pointer  displays  were  found  when  display  lag 
was  held  constant.  A  faster  display  response  time  in  the  LCD  moving  pointer  display  improved 
accuracy  at  maintaining  a  specified  vertical  velocity,  but  did  not  affect  workload  (as  measured  by 
pitch  rate).  The  same  effect  would  be  expected  with  the  mechanical  moving  pointer  design,  but  this 
result  cannot  be  generalized  to  the  moving  tape  design  without  further  verification.  Note  that  the 
same  performance  differences  may  not  be  replicated  in  an  operational  setting,  since  the  tasks  in  the 
current  study  were  more  VVI-intensive  than  most  operational  uses  of  the  display. 

Subjective  data  were  consistent  with  these  objective  results  in  that  the  moving  tape  was  the 
least  preferred,  particularly  for  any  operational  uses  that  involved  interpreting  the  display  under 
dynamic  conditions  (e.g.,  determining  trend).  However,  display  legibility,  including  size  and 
design  of  markings,  display  layout,  pointer  design/visibility  and  the  ability  to  read  the  displays 
under  static  conditions  were  generally  acceptable  for  all  displays.  Display  brightness  and  contrast 
were  also  acceptable  for  the  simulator  lighting  conditions;  however,  it  is  emphasized  that  the 
current  evaluation  did  not  include  a  comprehensive  lighting  evaluation.  One  of  the  goals  of  the 
study  was  to  evaluate  possible  failure  indications  for  the  prototype  LCD  VVI.  Suggestions 
included  (1)  removal  of  the  pointer  from  the  display,  (2)  an  “X"  displayed  on  the  face,  (3)  an  off 
flag,  (4)  actuation  of  several  pointers  for  a  “fan"  indication  and  (5)  an  INS  failure  indication. 

On  the  basis  of  the  objective  and  subjective  results,  it  can  be  concluded  that  either  moving 
pointer  display  would  be  preferable  and  would  provide  equal,  if  not  improved,  performance  over 
the  current  moving  tape  display.  The  LCD  VVI  should  not  be  incorporated  until  the  hardware 
problems  experienced  in  the  study  are  resolved  and  display  reliability  is  shown  to  be  acceptable. 
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RECOMMNDATIONS 


a.  Since  no  differences  were  found  between  the  LCD  and  mechanical  moving  pointer  designs, 
if  the  F-16  SPO  determines  the  need  to  replace  the  current  VVI,  it  is  recommended  that  a  moving 
pointer,  semicircular  display  format  be  incorporated. 

b.  Prior  to  incorporating  an  LCD  type  vertical  velocity  indicator,  such  as  was  evaluated  in  the 
current  study,  efforts  should  be  taken  to  verify  its  reliability  in  a  flight  test  environment. 

c.  A  thorough  lighting  evaluation  that  investigates  the  acceptability  of  display  luminance  and 
contrast  in  the  entire  range  of  expected  operational  lighting  conditions  should  be  performed  prior  to 
incorporating  the  LCD  display  into  any  aircraft. 
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Appendix  A 
Pilot  Questionnaires 
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VVI:  Moving  Tape 


Pilot: 

The  following  questions  address  various  features  of  the  moving  tape  vertical  velocity 
indicator.  Respond  to  the  following  questions  by  checking  the  blank  or  providing 
comments,  as  appropriate. 

PART  1.  DISPLAY  DESIGN  AND  OPERATION. 

1.  Rate  the  following. 

Too  large  About  right  Too  small 

a.  Size  of  numbers: 


b.  Size  of  graduation  marks:  _  _  _ 

c.  Spacing  between  graduation  marks:  _  _  _ 

Too  high  About  right  Too  low 

2.  Contrast  between  marking/pointer 

and  display  background  _  _  _ 

3.  Were  you  able  to  set  display  brightness  to  an  acceptable  level? 

Yes _  No _ 


4.  Using  the  following  scale,  rate  the  following  items: 

A  =  Completely  Acceptable  D  =  Moderately  Unacceptable 

B  =  Moderately  Acceptable  E  =  Completely  Unacceptable 

C  =  Borderline 

A  B  C  D  E 

a.  Ability  to  quickly  read  displayed 

value.  _  _  _  _ 


b.  Overall  scaling  of  display 

c.  Indication  of  zero  vertical  velocity. 
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5.  These  questions  address  display  dynamics  and  operation.  Rate  the  following: 

A  B  C  D  E 

a.  Ease  of  interpreting  the  display 

under  changing  conditions.  _  _  _  _  _ 

b.  Ease  of  interpreting  the  display 

under  relatively  static  conditions.  _  _  _  _  _ 

c.  Quality  of  trend  information.  _  _  _  _  _ 

d.  Display's  ability  to  enhance  your 

control  of  the  aircraft.  _  _  _  _  _ 

e.  Overall  acceptability  for  use  in  the  _  _  _  _  _ 

F-16. 

g.  Is  the  tic  mark  at  -750  fpm  useful  for  approach  and  landing? 

Yes _  No _ 

h.  The  display  response  is:  Too  fast  About  right  Too  slow 

6.  Does  the  display  scale  present  information  at  sufficient  precision? 

Yes _  No _ 

If  no,  explain. _ 


7.  Provide  any  comments  regarding  the  operational  utility  of  the  display: 
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VVI:  Mechanical  Semicircular 


Pilot: 

The  following  questions  address  various  features  of  the  mechanical  semicircular  vertical 
velocity  indicator.  Respond  to  the  following  questions  by  checking  the  blank  or 
providing  comments,  as  appropriate. 

PART  1.  DISPLAY  DESIGN  AND  OPERATION. 

1.  Rate  the  following. 

Too  large  About  right  Too  small 

a.  Size  of  numbers: 


b.  Size  of  graduation  marks:  _  _  _ 

c.  Spacing  between  graduation  marks:  _  _  _ 

Too  high  About  right  Too  low 

2.  Contrast  between  marking/pointer 

and  display  background  _  _  _ 

3.  Were  you  able  to  set  display  brightness  to  an  acceptable  level? 

Yes _  No _ 


4.  Rate  the  following  items  with  the  scale  below: 

A  =  Completely  Acceptable  D  =  Moderately  Unacceptable 

B  =  Moderately  Acceptable  E  =  Completely  Unacceptable 

C  =  Borderline 

A  B  C  D  E 

a.  Ability  to  quickly  read  displayed  _  _  _  _  _ 

value. 

b.  Overall  scaling  of  display  _  _  _  _  _ 

c.  Location  of  number  labeling  on  scale  _  _  _  _  _ 

d.  Indication  of  zero  vertical  velocity.  _  _  _  _  _ 

e.  Legibility  of  pointer  _  _  _  _  _ 
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5.  These  questions  address  display  dynamics  and  operation.  Rate  the  following: 

A  B  C  D  E 

a.  Ease  of  interpreting  the  display 

under  changing  conditions.  _  _  _  _  _ 

b.  Ease  of  interpreting  the  display 

under  relatively  static  conditions.  _  _  _  _  _ 

c.  Quality  of  trend  information.  _  _  _  _  _ 

d.  Display's  ability  to  enhance  your 

control  of  the  aircraft.  _  _  _  _  _ 

e.  Overall  acceptability  for  use  in  the  _  _  _  _  _ 

F-16. 

6.  Would  a  tic  mark  at  -750  fpm  be  useful  for  approach  and  landings? 

Yes _  No _ 

7.  The  display  response  is:  Too  fast  About  right  Too  slow 

8.  Does  the  display  scale  present  information  at  sufficient  precision? 

Yes _  No _ 

If  no,  explain. _ 


9.  Provide  any  comments  regarding  the  operational  utility  of  the  display: 
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Pilot: _  VVI:  LCD _ response 

The  following  questions  address  various  features  of  the  Liquid  Crystal  vertical  velocity 

indicator  with  the _ response.  Respond  to  the  following  questions  by  checking  the 

blank  or  providing  comments,  as  appropriate. 

PART  1.  DISPLAY  DESIGN  AND  OPERATION. 

1.  Rate  the  following. 

Too  large  About  right  Too  small 

a.  Size  of  numbers: 


b.  Size  of  graduation  marks:  _  _  _ 

c.  Spacing  between  graduation  marks:  _  _  _ 

Too  high  About  right  Too  low 

2.  Contrast  between  marking/pointer 

and  display  background  _  _  _ 

3.  Were  you  able  to  set  display  brightness  to  an  acceptable  level? 

Yes  No _ 


4.  Rate  the  following  items  with  the  scale  below: 

A  =  Completely  Acceptable  D  =  Moderately  Unacceptable 

B  =  Moderately  Acceptable  E  =  Completely  Unacceptable 

C  =  Borderline 

A  B  C  D  E 

a.  Ability  to  quickly  read  displayed 

value.  _  _  _  _ 


b.  Overall  scaling  of  display 

c.  Location  of  number  labeling  on  scale 

d.  Indication  of  zero  vertical  velocity. 

e.  Legibility  of  pointer 
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5.  These  questions  address  display  dynamics  and  operation.  Rate  the  following: 

A  B  C  D  E 

a.  Ease  of  interpreting  the  display 

under  changing  conditions.  _  _  _  _  _ 

b.  Ease  of  interpreting  the  display 

under  relatively  static  conditions.  _  _  _  _  _ 

c.  Quality  of  trend  information.  _  _  _  _  _ 

d.  Display's  ability  to  enahance  your 

control  of  the  aircraft.  _  _  _  _  _ 

e.  Acceptability  of  mechanization  of 

LCD  pointer  _  _  _  _  _ 

f.  Overall  acceptability  for  use  in  the 

F-16.  _  _  _  _  _ 

6.  Would  a  tic  mark  at  -750  fpm  be  useful  for  approach  and  landing? 

Yes _  No _ 

7.  The  display  response  is:  Too  fast  About  right  Too  slow 

8.  What  do  you  reccommend  for  a  failure  indication? 


8.  Does  the  display  scale  present  information  at  sufficient  precision? 

Yes _  No _ 

If  no,  explain. _ 

9.  Provide  any  comments  regarding  the  operational  utility  of  the  display: 
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Pilot: _  VVI:  LCD _ response 

The  following  questions  address  various  features  of  the  Liquid  Crystal  vertical  velocity 

indicator  with  the _ response.  Respond  to  the  following  questions  by  checking 

the  blank  or  providing  comments,  as  appropriate. 

1.  Rate  the  following  items  with  the  scale  below: 

A  =  Completely  Acceptable  D  =  Moderately  Unacceptable 

B  =  Moderately  Acceptable  E  =  Completely  Unacceptable 

C  =  Borderline 

A  B  C  D  E 

a.  Ability  to  quickly  read  displayed 
value. 


2.  These  questions  address  display  dynamics  and  operation.  Rate  the 
following: 

a.  Ease  of  interpreting  the  display 

under  changing  conditions.  _  _ 

b.  Ease  of  interpreting  the  display 

under  relatively  static  conditions.  _  _ 

c.  Quality  of  trend  information.  _  _ 

d.  Display's  ability  to  enhance  your 

control  of  the  aircraft. 


e.  Acceptability  of  mechanization  of 
LCD  pointer 


f.  Overall  acceptability  for  use  in  the 
F-16. 


3.  The  display  response  is:  Too  fast  About  right  Too  slow 
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4.  Provide  any  comments  regarding  the  operational  utility  of  the  display: 


PART  2.  COMPARISON 


1.  Rank  the  displays  in  order  of  overall  ease  of  interpretation.  (l=most 
preferred,  4=least  preferred). 

Moving  Tape  _ 

Mechanical  Semicircular  _ 

LCD  (slow  response)  _ 

LCD  (increased  response)  _ 

2.  Considering  everything,  rank  the  displays.  (l=most  preferred,  4  =  least 
preferred). 

Moving  Tape  _ 

Mechanical  Semicircular  _ 

LCD  (slow  response)  _ 

LCD  (increased  response)  _ 

3.  Provide  any  other  comments  on  the  displays. 
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Appendix  B 
Pilot  Comments 
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F-16  Vertical  Velocity  Indicator  Evaluation 

Pilot  Comments 


1.  General: 

Pilot  1.  The  moving  tape  gives  the  best  feel  for  climbs  and  descents.  The  only 
enhancement  might  be  for  adding  the  altitude  in  round  numbers  at  the  bottom  in  a  display 
similar  to  the  F- 1 1 1 .  However,  I  think  it  works  just  fine  the  way  it  is.  I  felt  very 
comfortable  with  the  moving  tape  during  three  years  instrument  flying  at  Ramstein. 

Pilot  2.  The  tape  lighting  appears  to  be  better  than  in  the  actual  aircraft. 

Pilot  5.  LCD  display  was  just  as  good  as  mechanical  if  it  had  more  of  an  analog 
movement  versus  the  incremental  one  tested.  The  continuous  analog  movement  provides 
better  trend  information. 

Pilot  6.  Need  larger  ADI  to  properly  use  high  fidelity  ADI. 

Pilot  7.  The  mechanical  semicircular  display  is  very  nearly  as  good  as  the  LCD  -  in  a 
way  it  is  better  in  that  it  can  display  increments  between  tic  marks  in  the  low  vertical 
velocity  regions.  An  aspect  of  the  LCD  I  liked  was  the  discrete  movement  of  the  pointer. 
This  called  attention  to  change  in  vertical  velocity,  where  analog  movement  may  be  slow 
and  insidious. 

Pilot  8.  750  fpm  is  rarely  exactly  what's  required  anyway.  Probably  just  used  to  round 
gage  anyway. 

Pilot  9.  The  mechanical  semicircular  was  the  smoothest  movement  display  and  easiest  to 
use  for  this  test.  It  was  the  easiest  for  precision  and  trend  information. 

Moving  Tape  VVI: 

Pilot  I.  One  improvement  would  be  to  put  a  separate  digital  display  at  the  bottom  of  the 
VVI  to  show  whole  numbers  representing  thousands  of  feet. 

Pilot  2.  With  the  current  dampening,  the  tape  is  excellent. 

Pilot  3.  Too  much  lag. 

Pilot  4,  Chasing  the  lag  was  a  nuisance. 

Pilot  5.  In  static  conditions,  read  a  VVI  value  is  very  easy.  In  dynamic  situations  it  is 
much  harder.  Trend  information  is  very  difficult  to  obtain  from  tape.  Pilot  must  look  at 
instrument  [directly  whereas  trend  can  be  picked  up  in  the  periphery  with  a  moving  pointer 
type  display! 

Pilot  6.  Between  0-1500  fpm  need  a  larger  scale. 
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Pilot  7.  The  tape  didn't  provide  good  global  situation  awareness  of  vertical  velocity.  I 
felt  forced  to  only  look  to  it  for  a  sort  of  digital  assessment  of  stable  vertical  velocity.  It 
was  confusing  to  look  at  while  it  was  moving. 

Pilot  9.  Overall  moderately  acceptable  for  instrument  conditions;  however,  the  pilot  still 
has  to  focus  on  the  actual  display  and  the  numbers  instead  of  noting  a  relative  indication  in 
the  peripheral  field  of  view  as  in  standard  VVIs. 

Pilot  10,  Using  the  tape  was  stressful  to  the  eyes  with  the  AOA  tape  moving,  ADI 
moving  and  VVI  tape  moving.  In  a  tight  spot  I  could  foresee  a  problem  with 
misinterpretation  of  the  ADI.  With  vertigo  induced  the  eyes  would  probably  get  into  a 
fluttering  (unable  to  focus)  that  may  not  be  recoverable. 

2.  Mechanical  Semicircular  VVI: 

Pilot  2.  Needs  more  light  at  the  zero  mark. 

Pilot  3.  Needs  faster  reaction. 

Pilot  4.  This  seemed  very  responsive  and  accurate. 

Pilot  5.  Grad  marks  are  small  but  OK  for  intended  purpose. 

This  true  analog  was  better  than  digitally  presented  analog  display.  A  turbulent  study 
should  be  made.  Even  though  this  is  slow  response,  it  seemed  better  [than  the  LCD] 
because  of  true  analog  movement  versus  digital  incremental  movement. 

Pilot  6.  Excellent  VVI. 

Pilot  7.  Given  lighting  didn’t  allow  truly  precise  reading  of  the  display.  If  lighting  were 
improved,  I  think  I  would  score  the  display  higher/more  acceptable. 

Pilot  8.  [sufficient  precision  is  provided]  If  looking  for  only  moderate  precision  as  is 
normally  the  case  in  an  aircraft  -  no,  if  you  really  want  to  nail  at  +/-3000  fpm  (only  good 
for  +/-  200  fpm  in  that  range. 

Pilot  9.  I  really  liked  this  display  and  the  analog  movement  of  the  pointer.  It  was  very 
easy  to  read  and  interpret. 

Pilot  10.  It  really  needs  better  backlighting  to  be  an  effective  instrument.  I  had  to  use 
map  light  to  flood  it  for  use. 

3.  LCD  (General): 

Pilot  1.  Pointer  should  go  away  for  failure  indication. 

Pilot  2.  Once  you  capture  a  desired  climb/descent  point,  any  deviation  is  much  more 
quickly  recognized  due  to  the  jump  in  the  needle.  Requires  less  concentration  on  VVI. 
Lighting  is  more  uniform  because  of  back  lighting.  The  numbers  are  easier  to  read. 

Failure  Indication:  LCD  pointer  blanking  out  of  view  or  an  X  midway  along  LCD  pointer. 


45 


Pilot  3.  Need  smaller  scale  up  to  1500  feet  per  minute 

Provides  intuitive  display  of  control/performance  relationship.  I  set  something  and  the  VVI 
goes  to  a  position--  fits  with  a  computerized  model. 

Pilot  4.  The  clicking  between  points  doesn't  seem  to  allow  for  precision.  You  ask  for 
750  fpm,  do  I  set  it  at  700  or  800  fpm?. 

Spacing  between  graduation  marks  too  small  at  higher  numbers. 

Failure  indication  should  be  Loss  of  needle. 

Pilot  5.  At  smaller  VVI  reading  -  a  bit  hard  to  read  exact  graduation.  Prob.  close 
enough  though.  750  is  not  a  magic  number  with  different  loadings  and  approach  speeds  of 
F-16s.  Failure  indication  should  be  off  flag  like  ADI. 

Pilot  6.  INS  failure  light  for  failure  indication. 

The  pointer  is  on  backwards.  Shouldn't  the  small  end  be  the  "pointer"  end? 

Pilot  7.  Failure  indication:  Turn  off  the  needle  or  turn  on  the  XXX  needles. 

Pilot  8.  Failure  indication,  flag  covering  display  or  maybe  a  "fan  indication" 

[Sufficient  precision]  except  if  you  want  to  nail  a  high  rate  of  climb/descent. 

Pilot  9.  Failure  indication.  Total  blank  indicator  or  a  bar  or  X  through  the  display. 

Pilot  10.  Failure  indication.  Pointer  disappears  if  possible  or  the  entire  LCD  goes  blank. 

4.  LCD  (Slow  Response)  VVI: 

Pilot  1.  I  like  the  quicker  response  better. 

Do  not  like  ratcheting. 

Pilot  5.  I  like  this  much  better  than  the  original  tape  of  the  F-16.  The  faster  response 
was  much  better.  Took  too  long  to  settle,  however,  should  be  evaluated  in  turbulence  for 
final  determination.  Analog  display  is  good.  The  response  time  is  the  only  question  I 
have.  Must  evaluate  this  option  in  turbulence  to  get  final  answer. 

Pilot  7.  The  difference  between  this  and  fast  response  display  is  slight  in  this  simulator, 
given  that  both  are  very  good,  the  slower  display  may  be  the  most  useful. 

Pilot  8.  Probably  better  than  the  fast  rate  for  approaches. 

Pilot  9.  The  discrete  movement  of  the  pointer  from  one  value  to  another  was  initially 
distracting,  but  after  a  couple  minutes,  I  didn't  notice  it  as  much.  Tough  to  do  750  fpm 
when  you  only  have  700  or  800  to  choose  from. 

Pilot  10.  I  felt  trend  information  was  too  slow  even  though  I  liked  the  positive 
indications.  It  also  being  slower  didn’t  catch  my  eye  as  quickly  as  the  previous  [LCDFR] 
indicator  (it  was  quick  and  very  positive,  which  match  my  control  inputs.) 
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5.  LCD  (Fast  Response)  VVI: 

Pilot  3.  Much  better  with  faster  response. 

Pilot  5.  Display  should  be  looked  at  in  turbulence  to  evaluate  response  time. 

Pilot  6.  I  found  the  display  readable  while  I  was  watching  the  ADI.  This  is  a  good 
display. 

Pilot  8.  May  be  too  sensitive  during  approaches  in  turbulence. 

Pilot  9.  The  discrete  movement  from  one  value  to  another  initially  got  my  attention,  but 
then  was  less  noticeable  after  a  few  minutes.  Good  display.  There  was  a  good  point  to  the 
ratcheting  and  that  was  when  I  noticed  it  moving,  it  gave  me  a  good  indication  that 
something  had  changed  which  required  an  action  on  my  part. 

Pilot  10.  I  think  the  way  this  is  mechanized  (it  shows  a  very  positive  change)  is  good.  I 
can  very  easily  adjust  100  fpm  and  set  it. 
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